In general, data processing systems comprise a central processing unit (CPU) that executes instructions that are fetched from a main memory. One method to improve the performance of the CPU is to use cache memory. Cache memory is high speed memory that works in conjunction with the CPU and the main memory to provide the necessary data to the CPU. With this architecture, a higher response time is possible than if the CPU fetches all instructions and operands directly from main memory.
The improved performance is possible because the cache contains the data that the CPU is most likely to request in the next bus cycle. The cache is typically also much faster than the main memory, therefore, the cache can usually provide the data required by the CPU much faster than the main memory. Part of the methodology used to load data into the cache is to predict and store the data that is frequently used by the CPU and is likely to be used by the CPU in the next bus cycle. When the cache contains data requested by the CPU, this is referred to as a cache hit.
However, it is not always possible to predict and store the necessary data in the cache before the CPU requests the data. If the cache does not contain the information requested by the CPU, then this is referred to as a cache miss. On a miss, the data is loaded from the main memory into a fill buffer and is provided to the CPU. On the next miss, the data in the fill buffer is then generally loaded into the cache in anticipation that the CPU may request the data in an upcoming bus cycle. This process continues throughout the operation of the processor.
During each cache miss, the data in the fill buffer is loaded into the cache. The transfer from the fill buffer into the cache may overwrite a valid piece of data in the cache. The data that is overwritten, however, may be the data that is requested by the CPU on the next bus cycle. In such a situation, on the next bus cycle the cache will be requires to fetch the data that was just overwritten in the previous cycle. Such situations reduce the efficiency of the use of a cache.
The loss in efficiency is further compounded by the time it takes to fill the fill buffer with data from the main memory. To increase the likelihood of a hit on the next bus cycle, it is common for the fill buffer to be filled with a large data stream such as 4 long words, which is 128 bits of data. However, if the next bus cycle only requires 16 or 32 bits of data, then most of the time spent to load the entire 128 bits is wasted.
By now it should be appreciated that it would be advantageous to provide an apparatus and method for improving the efficiency of a cache.